xxhash-addon

Yet another xxhash addon for Node.js

Platform	Build Status
Windows
Debian

Overview

xxhash-addon is a native addon for Node.js (>=8.6.0) written using N-API. It 'thinly' wraps xxhash v0.8.0, which has support for a new algorithm XXH3 that has showed to outperform its predecessor.

IMPORTANT: As of v0.8.0, XXH3 and XXH123 are now considered stable. Rush to the upstream CHANGELOG for the formal announcement! xxhash-addon v1.4.0 is packed with xxhash v0.8.0.

Features

• xxhash-addon exposes xxhash's API in a friendly way for downstream consumption (see the Example of Usage section).
• Covering all 4 variants of the algorithm: XXH32, XXH64, XXH3 64-bit, XXH3 128-bit.
• Supporting XXH3 secret.
• Consistently producing canonical (big-endian) form of hash values as per xxhash's recommendation.
• The addon is extensively sanity-checked againts xxhash's sanity test suite to ensure that generated hashes are correct and align with xxhsum's (xxhsum is the official utility of xxhash). Check the file xxhash-addon.test.js to see how xxhash-addon is being tested.
• Minimal dependency: the package does not depend on any other npm packages.

Installation

npm install xxhash-addon

Note: This native addon requires recompiling. If you do not have Node.js building toolchain installed then you must install them first:

• On a Windows machine

npm install --global --production windows-build-tools

• On a Debian/Ubuntu machine

sudo apt-get install python g++ make

• On a RHEL/CentOS machine

If you are on RHEL 6 or 7, you would need to install GCC/G++ >= 6.3 via devtoolset- for the module to compile. See SCL.

• On a Mac

Install Xcode command line tools

In comparison to other implementations/bindings

• node-xxhash

Although node-xxhash offers incredible compability as it requires Node.js >= 4.0.0, it does not seem to produce consistent hash values. For example, given a seed value of 0xDEADBEEF and the string 'hello', node-xxhash's XXH32 produces 2717969635 as the string's hash, which is in big-endian form, whilst its XXH64 produces cd6d9204aaad5b0c, which is in little-endian form.

• xxhashjs

This fantastic project is a pure JavaScript implementation of the algorithm. However it has not been updated for years.

Example of Usage

// Construct a new hasher and seed it 0 (default seed value)
// Note: constructors can take either JS Number or Buffer as their argument
const { XXHash32, XXHash64, XXHash3 } = require('xxhash-addon');
const hasher32 = new XXHash32(0);
// const hasher32 = new XXHash32(); // equivalent to the previous call
// const hasher32 = new XXHash32(Buffer.alloc(4)); // equivalent to the previous call, too

// Hash a string
const salute = 'hello there';
const buf_salute = Buffer.from(salute);
console.log(hasher32.hash(buf_salute));

// Digest a byte-stream (hash a buffer piece by piece)
hasher32.update(buf_salute.slice(0, 3));
console.log(hasher32.digest());
hasher32.update(buf_salute.slice(3));
console.log(hasher32.digest());

// Reset the hasher for another hashing
hasher32.reset();

// Using secret for XXH3
// Same constructor call syntax, but hasher switches to secret mode whenever
// it gets a buffer larger than 135 bytes
const hasher3 = new XXHash3(require('fs').readFileSync('package-lock.json'));

API reference

XXHash32

(constructor) XXHash32([Number or 4-byte Buffer])
(constructor) XXHash32() - using default seed value of 0
XXHash32.update([Buffer]) - updates internal state for stream hashing
XXHash32.digest() - produces hash of a stream
XXHash32.reset() - resets internal state. You can use this rather than creating another hasher instance

XXHash64

(constructor) XXHash64([Number or 4-byte Buffer or 8-byte Buffer])
(constructor) XXHash64() - using default seed value of 0
XXHash64.update([Buffer]) - updates internal state for stream hashing
XXHash64.digest() - produces hash of a stream
XXHash64.reset() - resets internal state. You can use this rather than creating another hasher instance

XXHash3

(constructor) XXHash3([Number or 4-byte Buffer or 8-byte Buffer]) - using seed
(constructor) XXHash3() - using default seed value of 0
(constructor) XXHash3([longer-than-135-byte Buffer]) - using secret
XXHash3.update([Buffer]) - updates internal state for stream hashing
XXHash3.digest() - produces hash of a stream
XXHash3.reset() - resets internal state. You can use this rather than creating another hasher instance

XXHash128

(constructor) XXHash128([Number or 4-byte Buffer or 8-byte Buffer]) - using seed
(constructor) XXHash128() - using default seed value of 0
(constructor) XXHash128([longer-than-135-byte Buffer]) - using secret
XXHash128.update([Buffer]) - updates internal state for stream hashing
XXHash128.digest() - produces hash of a stream
XXHash128.reset() - resets internal state. You can use this rather than creating another hasher instance

Licence

The project is licensed under BSD-2-Clause.

Changelog

v1.5.0

Dependencies

• Upgrading xxhash to v0.8.1

Features

• Support for TypeScript - https://github.com/ktrongnhan/xxhash-addon/pull/14